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Claims 

1 . A color image pickup device that is characterized by being equipped with an image 
pickup element with an offset sampling structure, possessing 4 types of color filters, with the color 
filters repeating in a 2 pixel period different from one another in the vertical direction, and with 
different color filters repeated offset by a 2 pixel period in the horizontal direction and by one pixel 
in the vertical direction, and 

a memory in which the output from the aforementioned image pickup element is temporarily 
stored, 

which are constituted so that an image is obtained by reading the required information from the 
aforementioned memory and performing digital signal processing. 



" [Numbers in the margin indication pagination of the original foreign language text.] 
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2. A color image pickup device that is characterized by being equipped with an image 
pickup element with an offset sampling structure, possessing 4 types of color filters, with the color 
filters repeating in a 2 pixel period different from one another in the vertical direction, and with 
different color filters repeated offset by a 2 pixel period in the horizontal direction and by one pixel 
in the vertical direction, 

an optical low-pass filter, constructed possessing an optical member that splits incident light rays 
into two light rays separated by a distance D in directions that form an angle 0in the clockwise 
direction and counter-clockwise direction relative to the scanning direction of the aforementioned 
image pickup element, so that the distance D fulfills the condition 

1* 6PhPv/l2Pbsi A0+Pvcos0!£D£2. 4PhPv/ 

where Ph is the pitch in the horizontal direction of the aforementioned image pickup element and 
Pv is the pitch in the vertical direction, and 

a memory in which the output from the aforementioned image pickup element is temporarily 
stored, 

which are constituted so that an image is obtained by reading the required information from the 
aforementioned memory and performing digital signal processing. 

3. The color image pickup device disclosed in Claim 1 or 2, which is characterized by 
including a gain adjustment means that adjusts the gain of the output from the aforementioned 
image pickup element so that the output from the aforementioned 4 types of color filters is uniform 
with achromatic subject image, and 

a signal formation means that forms a RGB signal from the output of the aforementioned 4 types of 
color filters by computing a 3 line x 4 column matrix in which the sum of the coefficients of any 
two columns is equal to the sum of the coefficients of the remaining two. 

Detailed explanation of the invention 
[0001] 

Industrial application field 

The present invention pertains to a color image pickup device equipped with a solid state 
image pickup element comprised of a multiplicity of light receiving elements arrayed in 
two-dimensions, and in particular pertains to a color image pickup device that performs digital 
signal processing using a memory. 
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[0002] 
Prior art 

With the rapid advancement of semiconductor technology in recent years, it has become a 
certainty that signal processing will be performed digitally in the field of video signal processing. 
This is due to the advantages of digital signal processing in terms of increased IC integration and 
precision, the ease with which a variety of adjustments can be made, and the excellent S/N ratio, 
etc. compared with analog signal processing. 

[0003] 

Consequently, based on the strengths of image pickup elements, such as small size, light 
weight, and high reliability, etc., a variety of systems have been developed for digital applications. 
CCD image pickup elements, with a color filter array like that shown in Figure 9, have become the 
most commonly used of these today as image pickup elements in video cameras and electronic still 
cameras from the aspects high resolution, good S/N ratio, and signal processing ease, etc. 

[0004] 

As is apparent from Figure 9, this image pickup element possesses color filters in which the 
colors repeat in a 2 pixel period different from one another in the horizontal direction. Thus, said 
filter array comprises lines in which magenta Mg and green Gr are repeated, and lines in which 
cyan Cy and yellow Ye are repeated. Additionally, Mg and Gr constitute an offset sampling array 
and Cy and Ye constitute a square sampling array. Further, there are also arrays in which this is 
reversed, with Mg and Gr in a square sampling array and Cy and Ye in an offset sampling array. 

[0005] 

Figure 10 is a chart that shows the color carrier characteristics in a spatial frequency plane 
(f H , fv) of an image pickup element color filters arrayed as described above. In Figure 10, the 
arrows represent the carriers for the various colors Mg, Gr, Cy, and Ye, and the direction of each 
arrow represents the phase relationship between the various colors. In this figure, fn shows the 
frequency in the horizontal direction and fv shows the frequency in the vertical direction. 

[0006] 

A method called pixel mixture (or field readout) is commonly used as the method of 
reading the signals from these image pickup elements. This is a method in which the 
photoelectrical^ converted signal charges are read mixing two lines at a time. With this method, 
the signal charges are read for field A as An_i, An, A n+J , . . ., e.g., as shown in Figure 9. Field B is 
also read as B fr i, B n , B n+ i, . . . 
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[0007] 

As a result, pixel signals (Mg+Cy) and (Ye+Gr), and (Cy+Gr) and (Mg+Ye) are alternately 
output from this image pickup element. Now, if the signals obtained from the image pickup 
element during use are referred to as 

Ci= Mg+Cy 

C 2 = Ye+Gr 

C 3 = Cy+Gr 

C 4 = Mg+Ye 

the set at the position of the spatial center of gravity when these four signals are obtained can be 

thought of as having a square sampling structure comprising Ci, C2, C3, C4, as shown in Figure /; 

1 1 . A color image is then obtained by forming a brightness signal and color signal for each field 

based on this pixel signal. 

[0008] 

Problems to be solved by the invention 

However, when pixel mixture is performed using an image pickup element with a color 
filter array like that shown in Figure 9, the following kinds of problems arise. First, if this image 
pickup element is used to pick up static images in an electronic still camera, etc., only one of the 
fields can be imaged due to pixel mixing, so that so-called high-resolution frame images cannot be 
obtained. 

[0009] 

Second, when pixel mixture is performed, a brightness signal and color signal are formed 
from the image pickup element using a sampling structure like that shown in Figure 1 1, as 
described above, but to do this, two lines of data must be used, skipping one line between. Since 
data are thus combined for two spatially separated points, the correlation in the vertical direction is 
diminished, which deteriorates the resolution, especially in the vertical direction. 

[0010] 

Third, since the vertical correlation is diminished in the color signal, as described above, 
the color moire effect becomes more apparent. This occurs for the reasons shown below. Figure 12 
is a chart that shows the color carrier characteristics in a spatial frequency plane (fpi, fv) of an 
image pickup element based on a structure like that shown in Figure 11. The arrows in Figure 12 
represent the carriers for the various colors Ci, C2, C3, C4, and the direction of each arrow 
represents the phase relationship between the various colors. In addition, Ph represents the pitch in 
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the horizontal direction of the image pickup element and Pv represents the pitch in the vertical 
direction. 

[0011] 

When this is compared with the color carrier characteristics in the spatial frequency plane 
(fn, fv) shown in Figure 10, it can be seen that color carriers arise at a lower frequency (0, Vi Pv) in 
the vertical direction in Figure 12. This corresponds with 240TV in a NTSC video camera. Due to 
the action of this color carrier, color moire effect occurs at a lower frequency in the vertical 
direction. It can also be seen from Figure 10 that carriers also occur at the frequency (0, X A Pv). 
Thus, due to the action of this color carrier, a color moire effect will also occur even when 
photographing subjects with fine achromatic patterns. 

[0012] 

Fourth, because of the characteristics of the optical low-pass filter used to prevent the 
occurrence of color moire effect, adequate resolution could not be obtained. Namely, it can be seen 
in both Figures 10 and 12 that a color carrier occurs at the position of the frequency QA Ph, 0) in the 
horizontal direction. This color carrier is supposed to be suppressed by the optical low-pass filter 
positioned in front of the image pickup element, but meanwhile, using said optical low-pass filter 
invites deterioration of the resolution in the horizontal direction. Therefore, optical low-pass filters 
had to be designed and picture quality determined in conventional video cameras with a trade-off 
between resolution and the color moire caused by the color carriers. 

[0013] 

This problem will be explained in detail using Figures 13-15. According to the sampling 
theorem, the brightness signal should be able to have a normal bandwidth at down to fn = Vi Ph. 
Consequently, an optical low-pass filter ideally will cut all frequency components greater than the 
Nyquist frequency fa = V2 Ph to 0 [zero] in order to prevent aliasing distortion in the brightness 
signal. 

[0014] 

However, since color carriers occur at the position of fn = V2 Ph, as described above, if an 
optical low-pass filter with the above-described frequency characteristics is used, a color moire 
effect will occur as the result of aliasing distortion in the color difference signal. Therefore, the 
frequency characteristics of the optical low-pass filter must be set so that the pass-band is lower 
than the frequency fn = V2 Ph at which a color carrier arises, as shown in Figure 13. 



[0015] 

The general configuration of a conventional optical low-pass filter that presumes the above 
is shown in Figure 14. In Figure 14, an optical low-pass filter 110 comprises a birefringence plate 
1 1 that splits incidental light rays in two at a distance of Ph in the horizontal direction, a phase 
plate 1 12 that circularly polarizes linearly polarized light, and a birefringence plate 1 13 that splits 
incident light rays in two at a distance of Ph/d in the horizontal direction. 

[0016] 

The transmission characteristic (MTF characteristic) of this optical low-pass filter is 
expressed by the following equation: 

Hi (fa, fv) = |cos {k! 2 • Ph f H ) • cos (;rPh f H )| ... (1) 

[0017] 

Figure 15(a) expresses this in the form of a graph, and Figure 15(b) expresses it as a 
characteristic on a spatial frequency plane (fa, fv), wherein dotted lines 1 14a, 1 14b, 1 15a, 1 15b are 
resonance frequencies that are trapped by the optical low-pass filter 1 10. Thus, it can be seen that 
the optical low-pass filter will trap the brightness signal carrier frequency fa = 1/Ph and the color 
difference signal carrier frequency fa = l/2Ph. 

[0018] 

In addition, in an optical low-pass filter 1 10 with characteristics like those described above, 
as can be seen from Equation (1) and Figure 15(a), the MTF is already dropped to -15dB at a 
frequency approximately 85% of the color difference signal carrier frequency fn = V2 Ph, making 
the MTF 0 [zero] at fa = l A Ph. Consequently, in past camera image pickup devices that use this 
kind of optical low-pass filter 1 10, no better than low resolution could be obtained at below the 
resolution limit of fa = V2 Ph. 

[0019] 

In addition, since frequency components over fa = V2 Ph cannot be adequately suppressed, 
as is clear from Figure 1 5(a), there were also problems with color moire effect occurring due to the 
resulting aliasing distortion in the color carrier. 

[0020] 

This invention addresses these kinds of problems and its purpose is to provide a color 
image pickup device that provides for the rationalization of a variety of processing through digital 
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signal processing, and which makes photography possible with good resolution and minimal color 
moire effect, even in still frame pictures. 

[0021] 

Means to solve the problems 

The color image pickup device of this invention is characterized by being equipped with an 
image pickup element with an offset sampling structure, possessing 4 types of color filters, with 
the color filters repeating in a 2 pixel period different from one another in the vertical direction, 
and with different color filters repeated offset by a 2 pixel period in the horizontal direction and by 
one pixel in the vertical direction, and a memory in which the output from the aforementioned 
image pickup element is temporarily stored, which are constituted so that an image is obtained by 
reading the required information from the aforementioned memory and performing digital signal 
processing. 

[0022] 

This invention is also characterized by being equipped with the aforementioned image 
pickup element and with an optical low-pass filter, constructed possessing an optical member that 
splits incident light rays into two light rays separated by a distance D in directions that form an 
angle 6 in the clockwise direction and counter-clockwise direction relative to the scanning 
direction of the aforementioned image pickup element, so that the distance D fulfills the condition, 

1 .6PhPv / |2Phsin0 + Pvcos9| < D < 2.4PhPv / |2Phsin0 + Pvcos6| (0 < 6 < 7d2) 
where Ph is the pitch in the horizontal direction of the aforementioned image pickup element and 
Pv is the pitch in the vertical direction, and a memory in which the output from the aforementioned 
image pickup element is temporarily stored, which are constituted so that an image is obtained by 
reading the required information from the aforementioned memory and performing digital signal 
processing. 

[0023] 

This invention is further characterized by possessing a gain adjustment means that adjusts 
the gain of the output from the aforementioned image pickup element so that the output from the 
aforementioned 4 types of color filters is uniform with achromatic subject image, and a signal 
formation means that forms a RGB signal from the output of the aforementioned 4 types of color 
filters by computing a 3 line x 4 column matrix in which the sum of the coefficients of any two 
columns is equal to the sum of the coefficients of the remaining two. 
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[0024] 
Function 

According to this invention, constituted as described above, by arranging 4 types of color 
filters disposed in an image pickup element in an offset sampling structure, offset 2Ph in the 
vertical direction, and 2Pv in the horizontal direction and Ph in the vertical direction, color carriers 
are minimized in the horizontal direction and in the vertical direction. 

[0025] 

Furthermore, by constituting an optical low-pass filter using an optical member that splits 
incident light rays into two rays in the clockwise direction or counter-clockwise direction relative 
to the scanning direction of the image pickup element, separated by a distance D, which fulfils a 
constant condition, a sufficient MTF characteristic can be obtained in the range below the limit 
resolution, and frequency components above the frequency of the limit resolution can be 
adequately suppressed. 

[0026] 

Additionally, by adjusting the output from the 4 types of color filters to yield uniform gain, 
and by calculating a matrix when forming the RGB signal from said output using a 3 -line x 4-row 
matrix in which the sum of the coefficients of any two rows in all the lines is equal to the sum of 
the coefficients of the remaining two, the occurrence of color carriers is further reduced in the 
vertical direction. 

[0027] 

Embodiments 

Embodiments of this invention will be described below, based on the attached figures. 
Figure 1 is a block schematic that shows a first example embodiment of the color image pickup 
device of this invention. In Figure 1, 1 is an image pickup element that converts light from the 
optical low-pass filter (not shown) to electrical signals, on which 4 types of color filters are arrayed 
as described below. 

[0028] 

An image pickup signal is read out one pixel at a time from the image pickup element 1 . 
The image pickup signal read from the image pickup element 1 undergoes processing, such as 
CDS (removal of reset noise or clock noise from the image pickup element 1 output signal) or 
AGC (input signal level correction), etc., in a pre-processing circuit 2, after which it is A/D 
converted by an A/D converter 3 at a timing synchronized with a readout clock. It is preferred that 



the A/D converter 3 have a linear characteristic for subsequent color processing, and that it 
perform A/D conversion at 8-bit or greater precision in consideration of quantization error. 

[0029] 

The A/D converted signal is temporarily written to a buffer memory 4 for 2-dimensional 
signal processing that will be performed later. The signal required for processing is then read out 
from this buffer memory 4. 

[0030] 

The signal corresponding to the brightness signal is sequentially read out from the buffer 
memory 4 and input to a low-pass filter 5 in sequence corresponding to the pixel array of the image 
pickup element 1. The signal input to the low-pass filter 5, after being limited to the desired 
bandwidth, is pressed as desired by a ^ converter 14 and an enhancer 15, to yield a brightness 
signal Yh with a high-band component. The brightness signal Yh, thus obtained with a high-band 
component, is integrated with a brightness low-band component Yl, obtained by a method 
described below, by a brightness signal formation circuit 16 and then output as a brightness signal 
Y. 

[0031] 

Meanwhile, signals corresponding to the color signals Mg, Gr, Cy, Ye are read out from 
the buffer memory 4 in the same manner as above, and converted to concurrent color signals 
Mg, Gr, Cy, Ye by four interpolation filters 6, 7, 8, 9. These 4 color signals are then input to an 
RGB converter 10 and converted to three primary color signals R, G, B. Further, this conversion is 
performed by the matrix calculation shown below. 

[0032] 
Equation 1 



R 








0 
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- A 


Gr 
Cy 


• • • (2) 









[0033] 

Wherein, Matrix A is a 3 -line x 4-row matrix optimized so that the spectral characteristics 
Mg(A), Gr(Z\ Cy(X), Ye(A) of Mg, Gr, Cy, and Ye in the image pickup element 1 are closer to the 
ideal spectral characteristics R(A), G(X), B(A), of R, G, and B specified in the NTSC standard. 
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[0034] 

Next, each of the three R, G, and B signals formed by the RGB converter 10 is 
white-balanced by a white balancer 1 1 by being converted to a form oR, G, /B. The RGB signal is 
then ^-converted by table conversion in a ^converter 12, after which it is input to a color difference 
matrix 13. The color difference matrix 13 produces the low-band component Y L described above 
and two color difference signals R-Y and B-Y by performing color difference conversion in 
accordance with the NTSC standard, as shown below. 



[0035] 
[Equation 2] 



Y 




0.30 0.59 


0.11 




R y 




R*Y 




0.70 -0.59 


-0.1 1 






... (3) 


B-Y 




♦0.30 -0.59 


0.89 
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[0036] 

The color difference signals R-Y, B-Y are given the specified band restrictions by 
respectively connected low-pass filters 18, 19, and then output. Additionally, after the brightness 
low-band component Yl is given the specified band restrictions by low-pass filter 17, it is 
integrated with the brightness signal with the high-band component Yh by the brightness signal 
formation circuit 16, and then output as the brightness signal Y. 



[0037] 

The integration by this brightness signal formation circuit 16 is performed according to a 
conversion equation shown below: 

Y = Y H -£(LPF17 )*Y h + £Y l (0<5<l) ...(4) 
Wherein, (LPF17 ) represents a low-pass filter with the same bandwidth as the low-pass filter 17, 
and (LPF17 ) * Yh represents performing the same bandwith restriction as the low-pass filter 17 
on the brightness signal with the high-band component Yh- 

[0038] 

This kind of conversion is performed because the low-band component Y L of brightness is 
the correct brightness in the spectral characteristic, it will not necessarily have a good S/N ratio, 
while, conversely, the brightness signal that contains the high-band component Yh will have a 
good S/N ratio, but its brightness is not necessarily correct in the spectral characteristic. 
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[0039] 

5 may be set constant at 1, here, in cases where the S/N ratio of the brightness low-band 
component Yl is good. In this case, processing in the brightness signal formation circuit 16 is 
equivalent to replacing the bandwidth portion that is the same as the brightness low-band 
component Y L in the brightness signal that contains the high-band component Y H with the actual 
brightness low-band component Yl. 

[0040] 

In addition, if the spectral characteristic of the brightness signal that contains the high-band 
component Yh is close enough to the actual brightness characteristic, 8 may be set constant at 0 
[zero]. In this case, absolutely no processing is performed by the brightness signal formation 
circuit 16, and it is the same as if the brightness signal that contains the high-band component Y H 
were simply output unaltered as the brightness signal Y. Further, the value for 5 may be changed 
according to the light level and color data of the subject, taking the S/N ratio and color 
reproducibility into consideration. 

[0041] 

Figure 2 is a drawing showing an example embodiment of the color filter array disposed in 
the image pickup element 1 described above. This color filter array comprises 4 colors: Mg, Gr, 
Cy, and Ye. The colors are disposed in an offset sampling structure, with the colors repeating 
differing from one another by 2 Ph in the vertical direction, and the colors repeating differing from 
one another being offset by 2 Pv in the horizontal direction and Ph in the vertical direction. 

[0042] 

Further, this color filter array does not necessarily have to be as shown in Figure 2, e.g., Mg 
and Cy, or Gr and Ye, may be switched. Furthermore, the array is not limited to these 4 colors, and 
the aforementioned 4 colors may be any combination of 4 colors selected from the above and, e.g., 
other colors, such as red Rd, blue Bl, and white Wh, etc. 

[0043] 

Figure 3 is a frequency characteristic chart showing the characteristics of the color carriers 
in the spatial frequency plane (fn, fv) of the image pickup element 1 in which this kind of color 
filter is disposed. In Figure 3, arrows represent the carriers for the various colors Mg, Gr, Cy, and 
Ye, and the direction of each arrow represents the phase relationship between the various colors. 
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[0044] 

When this frequency characteristic is compared with the frequency characteristics of past 
image pickup elements shown in Figures 10 and 12, it can be seen that no color carriers occur at the 
position of the frequency fv = X A Pv in the vertical direction in this embodiment. Therefore, it is 
difficult for color moire effects that are caused by aliasing from these color carriers to occur. 
Furthermore, since there are no color carriers on the fv axis, resolution can be obtained in the 
brightness signal at up to the theoretical resolution limit of fv = V2 Pv. 

[0045] 

Next, a second embodiment of the color image pickup device of this invention will be 
described, based on Figure 4. In Figure 4, components corresponding with the structural elements 
in Figure 1 are given the same legend, and since their operation is basically the same as it was in 
Figure 1, their detailed description will be omitted. 

[0046] 

In this embodiment, a gain adjustment circuit 20 is disposed at the step before the A/D 
converter 3, which gain adjustment circuit 20 is characterized by adjusting the gain of the image 
pickup signal obtained for each pixel Mg, Gr, Cy, Ye so that they are uniform with an achromatic 
subject. Further, while gain adjustment may be performed on the analog signal prior to A/D 
conversion, as shown in Figure 4, it may also be performed after A/D conversion in cases where 
the A/D converter 3 has adequate quantization accuracy. 

[0047] 

Performing this kind of gain adjustment not only prevents extraneous modulation 
components from piggy-backing on the brightness signal, but is also able to improve the color 
signal as shown below. 

[0048] 

Now, let the Matrix A in the conversion equation shown in Equation (2) equal 

A = (aij)(i=l,2,3;j = l,2,3,4) ...(5) 
Meanwhile, since the gain of each pixel Mg, Gr, Cy, Ye is adjusted to be uniform with an 
achromatic subject, the baseband components of Mg, Gr, Cy, and Ye can be expressed as 

Mg(f H , fv) = Gr (f H , fv) = Cy (f H , fv) = Ye(f H , fv) = #f H , fv) ... (6) 

using a given function $fk, fv) in the frequency space. 
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[0049] 

Wherein, if the baseband component of each color is given a sufficient bandwidth 
restriction by a two-dimensional filter, such as interpolation filters 6, 7, 8, 9, the carrier 
components at the position of frequency QA Ph, 0) become 

Mg(f H , fv) = Gr (f H , fv) = -#f H , fv), Cy (f Hj fv) = Ye(f Hj fv) = #f H , fv) ... (7) 

[0050] 

Consequently, the carriers in the RGB signal at this point are expressed, according to 
Equations (2), (5), (6), and (7), as 

R(f H , fv) = (-a n - ai2 + a i3 + a H ) ^fk, fv), 
G(fH, fv) = (-a2i - a 2 2 + a23 + a 2 4) ^(fk, fv), 
B(f H , fv) = (-a 3 i - a 32 + a 33 + a 34 ) #f H , fv) ... (8) 

[0051] 

According to this, if the sum of the coefficients of the 1 st row and 2 nd row in each line of 
Matrix A is equal to the coefficients of the 3 rd row and 4 th row, i.e., if the coefficient 

an + a i2 = a i3 + a i4 (i=l,2,3) ... (9) 

is true, then the carrier components in the RGB signal will be eliminated at this point (Vi Ph, 0), and 
no carrier components will arise in the color signal at this point. 

[0052] 

In other words, since the carrier components in the vertical direction of the color signal can 
be eliminated by constructing the Matrix A in the RGB converter 10a so that it meets the 
conditions shown in Equation (9), it thus becomes possible to diminish the color moire effect 
occurring in subj ects with fine achromatic patterns . 

[0053] 

Further, in cases where gain adjustment can be accurately performed at various color 
temperatures by the gain adjustment circuit 20, it is unnecessary to adjust the white balance with a 
white balance component 1 1 . 

[0054] 

Next, a third embodiment of the color image pickup device of this invention will be 
described. In this embodiment, an optical low-pass filter 50 constructed as shown in Figure 7(a) is 
used as the optical low-pass filter that is disposed in front of the image pickup element 1, as shown 
in Figure 5. Further, the structure in Figure 5 other than the optical low-pass filter 50 is the same as 
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the structure of the color image pickup device in the first embodiment shown in Figure 1. 
Additionally, the structure other than the optical low-pass filter 50, as shown in Figure 6, is the 
same as the structure of the color image pickup device in the second embodiment shown in Figure 
4. 

[0055] 

In Figure 7(a), the optical low-pass filter 50 comprises a first optical member made from a 
birefringence plate 51 that splits light rays incident from a direction 45° from the horizontal 
direction at a first distance di in two, a second optical member made from a birefringence plate 52 
that splits light rays incident from the horizontal direction at a second distance d2 in two, and a 
third optical member made from a birefringence plate 53 that splits light rays incident from a 
direction -45° from the horizontal direction at the first distance di in two. Incident light rays are 
directed to the trajectories shown in Figure 7(b) by the action of these birefringence plates 51-53. 

[0056] 

This optical low-pass filter 50 is constructed so that it meets the conditions shown by the 
following inequality with respect to the first distance di: 

1 .6 PhPv / | 2Phsin0 + Pvcostf | < di < 2.4 PhPv / | 2Phsin0 + Pvcostf | 
(O<0<^2) ... (10) 

[0057] 

Namely, aliasing due to color carriers, particularly the color moire effect, cannot be 
suppressed if the first distance di is less than the lower limit shown in Inequality (10). Conversely, 
adequate resolution cannot be obtained if it is greater than the upper limit. Consequently, it is 
necessary that the first distance di fulfill Inequality (10) in order to prevent color moire and obtain 
adequate resolution. 

[0058] 

Incidentally, the transmission characteristics (MTF characteristics) of this optical low-pass 
filter 50 are expressed by the following equation. 

H 2 (f H , fv) = | cos (7rd 2 f H ) ' cos (xNl • di (f H + fv)) • cos (ttA/2 • di (f H - fv)) I 

...(11) 
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[0059] 

Wherein, the following coefficient is true in a 640 horizontal x 480 vertical nominal pixel 
count image pickup element, which is the image pickup element used in the case, e.g., of a NTSC 
display screen with a 3:4 aspect ratio. 

Ph = Pv ... (12) 

Therefore, the first distance di and second distance d 2 in this embodiment are 

di = 2V2/3-Ph,d 2 = Ph/2 ... (13) 

[0060] 

The MTF characteristics that meet the conditions in this Equation (13) are represented by a 
graph like that shown in Figure 8(a). Additionally, the characteristics in a spatial frequency plane 
(f H , fv) are represented as shown in Figure 8(b). In Figure 8(b), the dotted lines 54a, 54b, 55a, 55b, 
56a, 56b are resonant frequencies that are trapped by the optical low-pass filter. Thus, it can be 
seen that the optical low-pass filter 50 traps all of the carrier frequencies in the brightness and both 
color difference signals. 

[0061] 

In addition, since frequency components higher than the resolution limit frequency fa = V2 
Ph can be sufficiently suppressed, as can be seen from Figure 8(a), aliasing distortion will not 
occur in the brightness. Consequently, color moire can thus be diminished. Furthermore, as is 
apparent from Equations (1 1), (12), and (13), and Figure 8(a), since MTF does not drop below 
-15db in the frequency range fa < V2 Ph, resolution can be ensured at up to nearly the resolution 
limit frequency fa = V2 Ph. 

[0062]. 

Meanwhile, in currently widely used image pickup elements in which the brightness signal 
sampling frequency is 4 times the propagation frequency fsc of the color difference signal, the 
following relationship is true. 

Ph=1.2Pv .(14) 
At this time, if the first distance di and second distance d 2 are 

di = Ph, d 2 = Ph/2 ... (15) 

then the same effects as the effects described above can be obtained. 

[0063] 

With this embodiment, as described above, on the one hand, high resolution up to the limit 
resolution can be obtained depending on the characteristics of the optical low-pass filter 50, while 
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on the other hand, color moire can be sufficiently suppressed by the action of the color filter array 
in the image pickup element 1 and the characteristics of the optical low-pass filter 50. 
Consequently, when designing the optical low-pass filter 50, it is no longer necessary to consider 
the tradeoff between resolution and color moire, so that design can prioritize resolution. Therefore, 
since the optical low-pass filter 50o can be made thin and compact, it becomes possible to make 
the overall image pickup optical system more compact. 

[0064] 

Further, it is possible in the first through third embodiments to output the resulting 
brightness signal Y and color difference signals R-Y, B-Y either interlaced or non-interlaced, 
regardless of the presence of a gain adjustment circuit 20. Non-interlaced output is advantageous 
for image compression, as in card cameras, etc., and is suited to computer input. Interlaced output 
can be used for video cameras and still/video cameras. At this time, a D/A converter that converts 
the output signal to an analog signal is necessary at the output terminal. 

[0065] 

The various processing described above, such as interpolation, bandwidth limiting, and 
RGB conversion, etc. are all linear processes. Consequently, since the effect is the same whether 
this processing is sequenced before or after, the hardware in which it is used can be constructed 
with these various processors placed in any order. Consequently, simply changing their order, they 
are still equivalent to the block diagrams described above, and are all included in this invention. 

[0066] 

Effect of the invention 

Since the color filter array disposed on the image pickup element has an offset sampling 
structure with colors repeating in a 2 pixel period different from one another in the vertical 
direction, and with different colors repeated offset by a 2 pixel period in the horizontal direction 
and by one pixel in the vertical direction, color carriers in the horizontal direction and vertical 
direction can be reduced, and the color moire effect in the horizontal direction and vertical 
direction caused by these color carriers can be diminished. Furthermore, since color carriers 
occurring at the resolution limit can be diminished, high resolution can be obtained up to the 
resolution limit. 

[0067] 

Furthermore, since the optical low-pass filter is constructed using optical members that 
split incident light rays into two light rays separated by a distance D that meets a constant 
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condition, in directions forming an angle 6 in the clockwise or counter-clockwise direction relative 
to the scanning direction of the image pickup element, sufficient MTF characteristics can be 
obtained in the range below the limit resolution and high resolution can be obtained up to the limit 
resolution, and it is possible to diminish the color moire effect caused by aliasing distortion of the 
color carriers occurring at above the frequency of the limit resolution. Consequently, it is no longer 
necessary to consider the tradeoff between resolution and color moire, so that design can prioritize 
resolution and high-resolution still frame images can be photographed. 

[0068] 

In addition, since the gain of the output from the image pickup element can be adjusted so 
that the outputs from the 4 types of color filters are uniform, and since the matrix calculation 
performed when forming an RGB signal from this output is performed using a 3 -line x 4-row 
matrix, in which the sum of the coefficients of any two rows is equal to the sum of the remaining 
two coefficients in all the lines, color carriers can be further decreased in the vertical direction, 
which has the effect of allowing for further decreases in color moire effect in the vertical direction, 
yielding even better quality images. 

[0069] 

In addition, since these processes can be performed digitally, there are advantages in 
uniform and stable performance, etc. 

Brief description of the figures 

Figure 1 is a block diagram showing a first embodiment of the color image pickup device 
of this invention. 

Figure 2 is a drawing showing the color filter disposition in the image pickup element that 
constitutes the color image pickup device of this invention. 

Figure 3 is a characteristic chart of the color carriers in the spatial frequency plane of the 
image pickup element that constitutes the color image pickup device of this invention. 

Figure 4 is a block diagram showing a second embodiment of the color image pickup 
device of this invention. 

Figure 5 is a block diagram showing a third embodiment of the color image pickup device 
of this invention. 

Figure 6 is a block diagram showing another configuration for the third embodiment of the 
color image pickup device of this invention. 

Figure 7 is a structural drawing of the optical low-pass filter that constitutes the color 
image pickup device of the third embodiment. 
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Figure 8 is a characteristic diagram for the optical low-pass filter that constitutes the color 
image pickup device of the third embodiment. 

Figure 9 is a drawing showing the filter disposition in an image pickup element that 
constitutes a conventional color image pickup device. 

Figure 10 is a characteristic diagram of color carriers in the spatial frequency plane of the 
image pickup element that constitutes a conventional color image pickup device. 

Figure 1 1 is a drawing showing the sampling structure in the image pickup element in 
Figure 9 during pixel mixture readout. 

Figure 12 is a characteristic diagram of color carriers in the spatial frequency plane based 
on the sampling structure in Figure 11. 

Figure 13 is a characteristic diagram showing the pass-bandwidth of a conventional optical 
low-pass filter. 

Figure 14 is a structural drawing of a conventional optical low-pass filter. 
Figure 15 is a characteristic diagram for a conventional optical low-pass filter. 

Reference symbols 



1 Image pickup element 

3 A/D converter 

4 Buffer memory 
10, 10a RGB converter 

20 Gain adjustment circuit 

50 Optical low-pass filter 

51, 52, 53 Birefringence plate 



54a, 54b, 55a, 55b, 56a, 56b Frequency trajectories trapped by the optical low-pass filter 
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Key: 1 Sensor 

2 Pre-processor 

4 Memory 

6, 7, 8, 9 Interpolation Filter 

10 RGB Converter 

1 1 White Balance 

12 ^Converter 

1 3 Color Difference Matrix 

14 ^Converter 

15 Enhancer 

16 Brightness Signal Formation Circuit 
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1 1 White Balance 

12 ^Converter 

1 3 Color Difference Matrix 

14 ^Converter 

15 Enhancer 

16 Brightness Signal Formation Circuit 
50 Optical LPF 
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[006 1] as (a) frkbfrizit.z. mm 
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Tte. ^ >HfflSEK2 0«**t«J:fe-f, SWjoMIK 
fi-fY. -feUft-^R-Y. B-Y£yWy?-V'-.X 
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